HIDDEN SUB-HYPERGROUP PROBLEM 
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Abstract. The Hidden Subgroup Problem is used in many quantum algorithms such as Simon's algo- 
rithm and Shor's factoring and discrote log algorithms. A polynomial time solution is known in case of 
abelian groups, and normal subgroups of arbitrary finite groups. The general case is still open. An effi- 
cient solution of the problem for symmetric group Sn would give rise to an efficient quantum algorithm 
for Graph Isomorphism Problem. We formulate a hidden sub-hypergroup problem for finite hypergroups 
and solve it for finite commutative hypergroups. The given algorithm is efficient if the corresponding 
QFT could be calculated efficiently. 



1. BACGROUND 

Peter Shor in his seminal paper presented efficient quantum algorithms for computing integer fac- 
torizations and discrete logarithms. These algorithms are based on an efficient solution to the hidden 
subgroup problem (HSP) for certain abelian groups. HSP was already appeared in Simon's algorithm 
implicitly in form of distinguishing the trivial subgroup from a subgroup of order 2 of . 

The efficient algorithm for the abelian HSP uses the Fourier transform. Other methods have been 
applied by Mosca and Ekert [12]. The fastest currently known (quantum) algorithm for computing the 
Fourier transform over abelian groups was given by Hales and Hallgren [7]. Kitaev [10] has shown us how 
to efficiently compute the Fourier transform over any abelian group (see also [9]). 

For general groups, Ettinger, Hoyer and Knill [5] have shown that the HSP has polynomial query 
complexity, giving an algorithm that makes an exponential number of measurements. Several specific 
non-abelian HSP have been studied by Ettinger and Hoyer [4], Rotteler and Beth [15], and Puschel, 
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Rotteler, and Beth [14]. Ivanyos, Mangniez, and Santha [9] have shown how to reduce certain non- 
abeUan HSP's to an abeUan HSP. The non-abehan HSP for normal subgroups is solved by Hallgren, 
Russell, and Ta-Shma [8]. 

As for the Graph Isomorphism Problem (GIP), which is a special case of HSP for the symmetric group 
Sn, Grigni, Schulman, Vazirani and Vazirani [6] have independently shown that measuring representations 
is not enough for solving GIP. However, they show that the problem can be solved when the intersection 
of the normalizers of all subgroups of G is large. Similar negative results are obtained by Ettinger and 
Hoyer [4]. At the positive side, Beals [3] showed how to efficiently compute the Fourier transform over 
the symmetric group Sn (see also [11]). 

Definition 1.1. (Hidden Subgroup Problem (HSP)). Given an efficiently computable function f : G ^ S, 
from a finite group G to a finite set S, that is constant on (left) cosets of some subgroup H and takes 
distinct values on distinct cosets, determine the subgroup H. 

An efficient quantum algorithms for abelian groups is as follows. 

Algorithm 1.2. (abelian HSP). 
1. Prepare the state 




1 



and measure the second register, the resulting state is 




1 



where c is an element of G selected uniformly at random. 



2. Compute the Fourier transform of the "coset" state above, resulting in 




where G denotes the Pontryagin dual of G, namely the set of homomorphisms p : G ^ C 



3. Measure the first register and observe a homomorphism p. 
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Note that the resulting distribution over p is independent of the coset cH arising after the first stage, as 
the support of the first register in (1). Thus, repetitions of this experiment result in the same distribution 
over G. Also by the principle of delayed measurement, measuring the second register in the first step can 
in fact be delayed until the end of the experiment. 

Algorithm 1.3. (non-abelian HSP, normal case) 1. Prepare the state X^^gc '^'^^ measure 

the second register \ f{g))- The resulting state is X^/igjj |c/i)|/(c/i)) where c is an element of G selected 
uniformly at random. As above, this state is supported on a left coset cH of H. 

2. Let G denote the set of irreducible representations of G and, for each p £ G, fix a basis for the 
space on which p acts. Let dp denote the dimension of p. Compute the Fourier transform of the coset 
state, resulting in 

E E -7=^EM'='^)i/''^'i)i/(^^)) 

pGGi<^j<dp Vl-«l-l'-l hen 

3. Measure the first register and observe a representation p. 

As before, one wishes the resulting distribution to be independent of the actual coset cH and depend 
only on the subgroup H. This is guaranteed by measuring only the name of the representation p and 
leaving the matrix indices unobserved. The fact that 0{log{\G\)) samples of this distribution are enough 
to determine H with high probability is proved in [8]. 

2. HYPERGROUP REPRESENTATIONS 

A finite hypergroup is a set K = {cq, ci, . . . , c„} together with a *-algebra structure on the complex 
vector space CK spanned by K which satisfies the following axioms. The product of elements is given 
by the structure equations 

* Cj = Y^n^jCk, 
k 

with the convention that summations always range over {0, 1, ... , n}. The axioms are 

(1) G M and n^j > 0, 

(2) Efe<, = l, 

(3) Co * Cj = Cj * Co = Cj, 
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(4) K* = K, ^ if and only if c* = Cj, 
for each < i,j,k < n. 

If c* = ci, for each i, then the hypergroup is called hermitian. If Cj * cj = Cj * Ci, for each then the 
hypergroup is called commutative. Hermitian hypergroups are automatically commutative. 

In harmonic analysis terminology, we have a convolution structure on the measure algebra M{K). This 
means that we can convolve finitely additive measures on K and, for x,y G K, the convolution Sx * Sy is 
a probability measure. Indeed 5^ * Scj{ck} = n^ j. We follow the convention of harmonic analysis texts 
and denote the involution hy x x (instead of a;*), and the identity element by e (instead of cq). For a 
function / : i4' — > C, and sets A,BCKwe put 

f{x*y) f{z){6x*Sy){z}, {x,y€K), 

zeK 

and 

A* B = \j{supp{5x 6y) : X e A,y e B}. 

A finite hypergroup K always has a left Haar measure (positive, left translation invariant, finitely additive 
measure) lu = luk given by 

oj{x} = {{6s * dx){e})~'^ {x€K). 

A function p : K ^ C is called a character if p{e) = l,p{x * y) — p{x)p{y), and p{x) — p{x). In contrast 
with the group case, characters are not necessarily constant on conjugacy classes. Let K he a. finite 
commutative hypergroup, then K denotes the set of characters on K. In this case, for p, e M{K) and 
/ G e{K), we put 

= X] /'(^)'"{^}' hp) = X] f{^)P{^)^{^} {P S 

xeK xeK 
Hence / = (/w)'. If H C K is a. subhypergroup (i.e. H = H and H* H C H), then (Djj = Xh-^ [2, 2.1.8], 
where the right hand side is the indicator (characteristic) function of 

H^ = {pek: p{x) = l{x& H)). 

If K/ H is the coset hypergroup (which is the same as the double coset hypergroup K/ /H in finite case 
[2, 1.5.7]) with hypergroup epimorphism (quotient map) q : K H/K [2, 1.5.22], then {K/Hy::^ H-^ 
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(with isomorphism map X'~^ X°<l) P, 2.2.26, 2.4.8]. Moreover, for each fj, G M{K), q^n * ujh) = Q'(m) [2, 
1.5.12]. We say that K is strong if ^ is a hypergroup with respect to some convolution satisfying 

where 

k{x) = k{p)p{x)7r{p} {xGK,ke f{k, tt)) 

is the inverse Fourier transform. In this case, for p,a G K, we have p G cr * if and only if ResnP = 
Resncr, where ResH : K ^ H \s the restriction map [2, 2.4.15]. Also H is strong and K/H^ ~ H [2, 
2.4.16]. Moreover K [2, 2.4.18]. 

Let us quote the following theorem from [2, 2.2.13] which is the cornerstone of the Fourier analysis on 
commutative hypergroups. 

Theorem 2.1. (Levitan) If K is a finite commutative hypergroup with Haar measure co, there is a 
positive measure w on K (called the Plancherel measure) such that 

Moreover supp{Tr) = K and 7r{p} = 7r{p} . In particular the Fourier transform ^ is a unitary map from 
e{K,Lo) onto£'^{k,TT). 

In quantum computation notation, 
where 



r{x) 



r(x) = (^Kx)|V{p})^ {xeK). 
peK 

When ii' is a group, t{x) = \k\^, for each x € K. It is essential for quantum computation purposes 
to associate a unitary matrix to each quantum gate, however, if we write the matrix of 'S naively using 
the above formula we don't get a unitary matrix. The reason is that, in contrast with the group case, 
the discrete measures on spaces are not counting measure. More specifically, when K is a group, 
e'^(K) = 0^g^ C, where as here t{K,u)) = 0^^^^ w{a;}5C and 1^{K) = 0^^^ Tr{p}iC. The exponent 
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I is needed to get the same inner product on both sides. If we use change of bases \x) = lo{x}^ \x) and 
\p)' — ^{p}^ Ip)) Fourier transform can be written as 

5 : \x)' ^ u;{x}i ^ p{x)Tr{p}i\p)' , 

and the corresponding matrix turns out to be unitary. 

There are not many finite hypergroups whose character table is known [Wil]. Here we give two classical 
examples (of order two and three and compute the corresponding Fourier matrix. 

ExEimple 2.2 (Ross). The general form of an hypergroup of order 2 is known. It is denoted by K = Z2{0) 
and consists of two elements and 1 with multiplication table 



* 


So 


Si 


So 


So 


Si 


Si 


Si 


9So + (1 - 9)6i 



and Haar measure and character table 








1 


Ll! 


1 


1 
e 


Xo 


1 


1 


Xi 


1 


-0 



When ^ = 1 we get K = Z2. The dual hypergroup is again Z2{6) with the plancherel measure 





Xo 


Xi 




e 

i+e 


1 

i+e 



The unitary matrix of the corresponding Fourier transform is given by 



d2 = 



^ 1 ^ 



1 



Example 2.3 (Wildberger). The general form of hypergroups of order 3 is also known. We know that 
it is always commutative, but in this case, the Hermitian and non Hermitian case should be treated 
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separately. Let if = {0, 1, 2} be a Hermitian hypergroup of order three and put Ui = u){i}, for « = 0, 1, 2. 
Then the multiphcation table of K is 





A,, 
"(1 


"1 






6o 




S2 


Si 


Si 


^6o + aiSi + P1S2 


JiSi + 72<52 


52 


S2 


71^1 + 72^2 


^So + (3261 + Q!2(52 



where 



71^2 r, 72^1 . 1 + 7ia;2 , 1 + 72^1 . 

P2 = — , ai = 1 , Q!2 = 1 72 = 1 - 71) 



and 71, oji and 0J2 are arbitrary parameters subject to conditions < 71 < 1, wi > 1, a;2 > 1, and 



1 + 71 W2 < Wi 

1 + (1 - 7i)a;i < u>2- 
The Plancherel measure and character table are given by 



where 



and 





TT 





1 


2 


Xo 


Sl 
t 


1 


1 


1 


Xi 


t 


1 


X 


z 


X2 


S3 
t 


1 


y 


V 



ai — 71 D q;i — 71 D 

^~ 2 ^^2 ^~ 2 '^2 

_ Q2 — 72 -D _ a2 — 72 -D 

^~ 2 2w^' ^~ 2 2w^ 



D = ^^(1 + 71^2 - 721^1)^ + 4720;! 



S2 = y 



Si = Z^ + 





+ 




- {y'z' 


x2 
+ — 


+ — 


U}2 




W2 






+ 


1 




— + 


^' 


UJl 


UJ2 


UJ2 




+ 


1 


-(ar2 + 


h 


^) 


C02 




U)i 
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. 22, 2, 2 /2i22i2n 

t = xv +y +z — [x +yz +v). 

Let TTi = 7r{xi} = Y ^^'^ "^ij — ^WjTTj , for i, j = 0, 1, 2, then the Fourier transform is given by the unitary 
matrix 

/ \ 

Woo Wio W20 



Wol XWii ZW21 



V 



^^02 yWi2 VW22 

One concrete example is the normaHzed Bose Mesner algebra of the square. In this case, u)i = l,u>2 = 
2, 7i = /3i = Q!i = a2 = 0, 72 = 1, and /32 = 5- A simple calculation gives D = 2,x = l,y = z = —l,v = 0, 
and if we put tti = we get 1^2 = \ and 773 = ^. In this case, the Fourier transform matrix is 



^3=2 



1 1 \/2 
1 1 -^/2 
V2 -V2 



In the non-Hermitian case, the multiplication table of K is 



* 


So 


61 


52 


5o 


So 


Si 


S2 


h 


Si 


jSi + (1 - 'y)S2 


aSo + 7^1 + 7(^2 


S2 


S2 


aSo + jSi + 7(52 


(1 -j)6i +7(52 



where 7 = and a is an arbitrary parameter with < a < 1. When a = 1, we get K = Z^. The 
dual hypergroup is again K and the Plancherel measure and character table are given by 





TT 





1 


2 


Xo 


Sl 
t 


1 


1 


1 


Xi 


S2 

t 


1 


z 


z 


X2 


S2 
t 


1 


z 


z 



where 



z = 



—a ± Wa"^ + 2a 



si=2 — uji{a^ + Of), S2 = wi — 1, t = ijJ\{2 — o? — a). 
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Put TTj = 7r{xi} and Wij = yfuJiW], for i,j = 0, 1,2, then the Fourier transform is given by the unitary 
matrix 



Woo WW W20 



^3 



Wqi ZWii ZW21 



\ 



J 



Wq2 ZW12 ZW22 

As a concrete example, let us put wi = a;2 = 2, 7 = 4 and a = i to get z = and tti = i 



5' 



7r2 = TTa = g. In this case, the Fourier transform matrix is 



1 V2 V2 

V2 - 



l+iVE -l-iVb 
4 4 



l-iV5 -l+iVE 
4 4 



Lemma 2.4. ief ii' 6e commutative and H be a sub-hypergroup of K and p G K, then the following are 
equivalent. 

(i) p e H^, 

(ii) T,mec*H i^{m}p{fh) ^ 0, for each c&K, 
{ii^) Emec*ff ^ 0' fo'^ so'^e c&K. 

Proof, (i) (ii) If p G H-^ and q : K ^ K/H is the quotient map, then given C& K, q{p * ujh) = qip) 
for p = 5cUi & M{K). But clearly 

Hence p{Sc*h)i^ = po q{5ci^) ^ 0, where the last equality is because p o g g {K/H)" and a character is 
never zero. 

{iii) {i) If p ^ then the multiplicative map po q should be identically zero on K/ H (otherwise 



it is a character and p G H-^). Hence '}2mec*H = p(^c*h)'^ = 0, for each C& K. 



□ 



3. HSHP 

In this section we give an algorithm for solving hidden sub-hypergroup problem (HSHP) for abelian 
(strong) hypergroups. This algorithm is efficient for those finite commutative hypergroups whose Fourier 
transform is efficiently calculated. It is desirable that, following Kitaev [10], one shows that the Fourier 
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transform could be efficiently calculated on each finite commutative hypergroup. This could be difficult, 
as there is yet no complete structure theory for finite commutative hypergroups (see chapter 8 of [2]). 



Definition 3.1. (Hidden Sub-hypergroup Problem (HSHP)). Given an efficiently computable function f : 

K ^ S, from a finite hypergroup K to a finite set S, that is constant on (left) cosets of some subhypergroup 
H and takes distinct values Ac on distinct cosets c* H, for cG K. Determine the subhypergroup H. 

Algorithm 3.2. (abelian HSHP). 

1. Prepare the state \xo) |0). 

2. Apply to the first register to get 



E^w^k)'io). 



3. Apply the black box to get 



and measure the second register, to get 



xeK 



J2 u,{m}i\m)'\Xe), 



where c is an element of K selected uniformly at random, and Ac is the value of f on the coset c* H. 
4- Apply ^ to the first register to get 

J2 Ew{mMp}^p(m)|p)'|Ac) = ^^E'^M^( E <^{m}p{m))\p)'\K) 
5. Measure the first register and observe a character p. 

Note that the resulting distribution over p is independent of the coset c* H arising after the first step. 
Also note that by Lemma 2.2, the character observed in step 3 is in H-^. 



Theorem 3.3. If the Fourier transform could be efficiently calculated on a finite commutative hypergroup 
K, then the above algorithm solves HSHP for K in polynomial time. 
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There ar a variety of examples of (commutative hypergroups) whose dual object is known. One might 
hope to relate the HSP on a (non-abelian) group G to the HSHP on a corresponding commutative 
hypergroup like G (see next example). The main difficulty is to go from a function / which is constant 
on cosets of some subgroup H < G to a function which is constant on cosets of a subhypergroup of G. 
The canonical candidate / fails to be constant on costs of < G. 

We list some of the examples of commutative hypergroups and their duals, hoping that one can get 
such a relation in future. 

Example 3.4. If G is a finite group, then G :— {G'^Y is a commutative strong (and so Pontryagin [2, 
2.4.18]) hypergroup [2, 8.1.43]. The dual hypergroups of the Dihedral group -D„ and the (generalized) 
Quaternion group Qn are calculated in [2, 8.1.46,47]. 

Example 3.5. If G is a finite group and H is a. (not necessarily normal) subgroup of G then the double 
coset space G//H (which is basically the same as the homogeneous space G/H in the finite case) is a 
hypergroup whose dual object is A{G, H) [2, 2.2.46]. It is easy to put conditions on H so that G/ /H is 
commutative. 

There are also a vast class of special hypergroups (see chapter 3 of [2] for details) which are mainly 
infinite hypergroups, but one might mimic the same constructions to get similar finite hypergroups in 
some cases. 
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